{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "参考：\n",
    "[大叔学ML第五：逻辑回归](https://www.cnblogs.com/zzy0471/p/logicregression.html)  \n",
    "[逻辑回归原理小结](https://www.cnblogs.com/pinard/p/6029432.html#!comments)  \n",
    "在损失函数求导时，似乎不需要除以样本（虽然对结果不影响，因为求得是极小值时的参数theta，而不是极小值是多少，但是跟刘建平博客推导不同）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hVVfbw8e9KIfTeITEgJUCQ3qVIB5FilxkH26CjDgjMODiIMiq2ERTGyqCIr+KoCIIISDWEIghICb1LD4pSpCbs948k/iAmJDe59+5z7lmf5/FJck1yV3LDXmevvfc6YoxBKaWU94TZDkAppZQdmgCUUsqjNAEopZRHaQJQSimP0gSglFIepQlAKaU8KsJ2AL4oW7asiY2NtR2GUkq5ypo1a340xpTL/LirEkBsbCyrV6+2HYZSSrmKiOzL6nEtASmllEdpAlBKKY/SBKCUUh6lCUAppTxKE4BSSnlUwBOAiLwnIskiknTZY/8Wka0iskFEpotIyUDHoZRS6krBmAG8D3TP9Nh8IN4Ycx2wHXgikAGs2PUTe378NZBPoZRSrhPwBGCMWQIcz/TYPGNMSvqH3wJVA/X8ly4ZRs5IottrSxi3YAfnU1ID9VRKKeUqTlgDuA+Yk93/FJGBIrJaRFYfO3bM528eFiZMeaAFXetW4NUF2+nxWiLLd/6Yn3iVUiokWE0AIjICSAE+yu5zjDETjDFNjTFNy5X73UnmXClfvCCv92/M5Puak3LJ0H/iSoZ8so4fT5/PY+RKKeV+1hKAiAwAegF/MEG6L2X7WuWYN6Qdj95Qg1kbDtFpTAIfr/qBS5f0tphKKe+xkgBEpDvwD6C3MeZMMJ+7YGQ4f+tWmzmD21K7YjGemLaR295ZwdYjJ4MZhlJKWReMbaAfAyuA2iJyQETuB14HigHzRWSdiLwd6Dgyq1G+GJ8MbMkrtzVg97HT9Bq/lBfmbOHMhZScv1gppUKABKn64hdNmzY1gegG+vOvF3hhzhY+XX2AKiUL8UyfenSqU8Hvz6OUUjaIyBpjTNPMjzthF5B1pYoU4OVbG/Dpg60oXCCc+yev5qH/t4bDJ87aDk0ppQJGE8BlmlcrzVeD2vL3brVZvC2ZzmMSeHfpHlJSL9kOTSml/E4TQCYFIsJ45IYazB/SnmbVSvPsrM30eWMZ6/b/Yjs0pZTyK00A2YgpU5hJ9zTjzT805sfT5+n35jKempHEyXMXbYemlFJ+oQngKkSEnvUrsWBoewa0iuXDb/fRaUwCX64/hJsWz5VSKiuaAHKhWMFIRvWuxxePtKFi8YL89ePvGTDpO/b9pA3mlFLupQnAB9dVLckXj7Th6Zvqsnbfz3R9dQmvL9IGc0opd9IE4KPwMOHeNtVYMLQ9netU4JV52+k5LpFvd/9kOzSllPKJJoA8qliiIG/8oTGT7mnG+ZRL3DnhW4Z9up7jv16wHZpSSuWKJoB8uiGuPPOHtOfhDtcyY91BOo75hk+/268N5pRSjqcJwA8KFQjn8e5xzB7clprli/L45xu4c8K3bD96ynZoSimVLU0AflSrQjE+GdiKl2+5ju3Jp+g5LpGX5m7l7AVdJFZKOY8mAD8LCxNubxbNomEd6NuoCm99s4uuryWweFuy7dCUUuoKmgACpHSRArxyWwP+N7AlBcLDuHfSdzz80RqOnjxnOzSllAI0AQRcy+plmDO4HX/rWouFW5LpNCaB95ftIVUXiZVSlmkCCIICEWE82rEm84a0o/E1pRj15Wb6vrGMjQdO2A5NKeVhmgCC6JoyRZh8bzP+c1cjjpw8R583ljJq5iZOaYM5pZQFmgCCTES4qUFlFg5rzx9bXsPkFXvpNCaBrzYc1gZzSqmg0gRgSfGCkTzTJ54vHm5DuWJRPDJlLfe+/x37j5+xHZpSyiM0AVjWILokMx5pw1O96vLdnuN0eTWBN7/ZyYUUvQuZUiqwNAE4QER4GPddX40Fw9rToVZ5Xp67jV7/SWTVnuO2Q1NKhbCAJwAReU9EkkUk6bLHSovIfBHZkf62VKDjcINKJQrx9t1NeHdAU349n8rt76zg8anr+VkbzCmlAiAYM4D3ge6ZHhsOLDTG1AQWpn+s0nWqU4H5Q9vxUPtrmbY2rcHcZ6v36yKxUsqvAp4AjDFLgMy1jD7A5PT3JwN9Ax2H2xQuEMHwHnHMGnQ91csV5e9T0xrM7UzWBnNKKf+wtQZQwRhzGCD9bXlLcTheXMXifPZgK168uT5bj5yix7hEXvl6G+cuaoM5pVT+OH4RWEQGishqEVl97Ngx2+FYERYm3Nk8hoXD2nNTg8q8vngnXV9dQsJ2b/4+lFL+YSsBHBWRSgDpb7NtlWmMmWCMaWqMaVquXLmgBehEZYtGMfb2hkz5cwsiwoUB763i0SlrSdYGc0qpPLCVAGYCA9LfHwDMsBSHK7W+tixzBrdlaJdazNt8lE5jEvhgxV5tMKeU8kkwtoF+DKwAaovIARG5H3gR6CIiO4Au6R8rH0RFhDOoU03mPdaOhjEleWrGJm5+cxlJB7XBnFIqd8RNWwubNm1qVq9ebTsMxzHG8OWGwzzz5WaO/3qee1pXY2jXWhSNirAdmlLKAURkjTGmaebHHb8IrHImIvRObzDXv0UMk5bvofOYBOYmaYM5pVT2NAGEkBKFInmub32m/aU1pYsU4KEP13L/5NXaYE4plSVNACGoUUwpZj7ahidvrMO3u3+iy6sJvPXNLi6muq/B3B3vrOCOd1bYDkOpkKQJIERFhIfxQNvqLBjannY1y/HS3K30Gr+U1Xu1wZxbaTJU/qarhCGucslCTPhTU+ZvPsqomZu49e0V3NksmuE94ihZuIDt8LKVMdCtTO+ImvHxJw+2shZTsHnxZ1bBpQnAI7rUrUDra8swfuEOJi7dw7zNRxnRsw43N66CiNgOT12FJkMVKJoAPKRIVARP9KxD30ZVGDF9I8M+W8/UNQd4rl8815Yraju8K2QMbl4c7DIP+MUK6j9TFRj6l+VBdSoVZ+pDrfnfd/t5cc4WeryWyEPtq/PwDTUoGBluOzyVSd1Kxa/42EvJUAWWHgTzuGOnzvP87C1M//4gsWUK82zfeNrW9HbPJafIPPvx4mxI+YceBFNZKlcsilfvaMhHD7RARLj73VUM+vh7kk9dpcHc/lWQOCbtrQqaTx5spYN/HuV1B1Wo77zSEpACoE2NtAZzb32zi7e+2cXibck83j2OPzSPISzsskXi/atgcm9IvQDhBWDATIhubi/wEGZjsNdZhrdoAlC/KRgZzpAutejTsDIjZyQx8oskPl9zgNH94qlXuUTaJ+1NTBv8TWra272J+UoAOuCoQMrrDiqv7LzSBKB+p3q5onx4fwtmrDvEc19tpvfry7i3dSxD6pyiyIn9EBYBl0ibAcS2tR2uVaEyMHhlwFNX0gSgsiQi9G1UhRtql+elr7cycekevlp2nFGRm+gWaaDJAGhwV56v/nXAUcGQ1wV0ryy8awJQV1WicCTP96vPLeFLGLEijAcvPEbnlDWMiixBVQ/X/kMtgXllwFNX0gSgcqVJo2Z8ub4fk87fwKspN9MlMYohhXZxb5tqRIb7vplMBxwVTHn9+wr1v0s9B6Byb/8q2JvIgdKtGbU6ggVbkomrWIzR/erT5JpSefqWbk8Abo9feUN25wB0BqByL7o5RDenKvDfuoZ5vzWYW85dzWP4R7c4ShSOvOJLchogdeD0Jk2czqAJQOWJiNCtXkXa1CjLq/O3M2nZHuZtOsKTN9alT8PKrm4w58vgpAOYcjNNACpfikZFMLJXXfo1qsKIL5J47JN1fLZmP6fOpVAoMjxkFkmVf4Ta4rnbaQJQfhFfpQTT/tKaKSv38fLcbZw+n0LlkgV9/0bp6wzEtg36CWMdnJTXaAJQfhMeJtzdKpZu9Sry7Fdb+HL9IQpGhhFbpkjuBlFtM+EqeUmQuvvLWaw2gxORISKySUSSRORjEcnDJaNymvLFC/KfuxrxwX3NMQa2HjnFY//7nmOnzl/9C7NqMxFEGc3WWlQr/VsPfh2gVCizNgMQkSrAIKCuMeasiHwK3Am8bysm5V/tapVj/dNdeWPxTt5O2MWirckM71GHO5tFX9lgLkNs27Qr/4wZgIU2E3e8s4LNh08G/Xl9Yfvq2R+lMk2szmC7HXQEUEhEIoDCwCHL8Sg/KxgZzrCutZkzuB11KhXnn9M3cuvby9mS1SAb3Tyt7NNxhLXyT8bgf+pcCiv3HA/5dsDK26weBBORwcBo4Cwwzxjzhyw+ZyAwECAmJqbJvn37ghtkqAvioqsxhmlrDzJ69hZOnL3I/ddX47HONSlcwP5SVOar2gwtqpUGnHHFmjlG27HZnomo3HPcQTARKQX0AaoBvwCficgfjTEfXv55xpgJwARIOwkc9EBDWZAXXUWEW5pUpWNceV6au5UJS3bz1YbDjOpdjy51KwTsefNC1wB0gPcCm5denYE9xphjACIyDWgNfHjVr1L+4+fe/rlVqkgBXrzlOm5pUpUR0zfy5w9W07VuBUb1rkflkoUC/vxZybw7xYmctoPG9vOr/LOZAH4AWopIYdJKQJ0AbfRzuUCXZywvujaLLc2sv7bl3aV7GLdwO53HJjC0Sy3uaR1LRB4azPmTlwc3PQ/hHbbXAP4F3AGkAN8DDxhjst0r6KlmcMEqz1g8eHVFGMfP8NSMJBZvO0adSsV5vl88jWLy1mBO5Y/T1hpU/mW3BqDdQJ0qcQwsGp1WnpHwtJ0xbYfZjiqgjDHMTTrCqC83kXzqPH9oEcPfu8VRolBkzl+s/E6v/ENHdgnA9jZQlZ2M8oyEe+bWiyJCj/qVWDC0Pfe0jmXKyh/oNCaBGesO4qYLFaXcQmcATuaQ8owtGw+cYMQXG9lw4ARta5bl2T7xxJYtYjsspVxHS0DKlVIvGT78dh///nobF1Iv8egNNXiwfXWiIsJth6aUa2gJKBTtX5W2VrB/le1I/o+fYwoPEwa0jmXhsPZ0qVuBsfO302NcIst3/eiX769CR1antvUk99XZP4Kp8saJnTMDGFOF4gV5o39jbmuSzFMzNtH/vyu5uXEVRvSsQ5miUX55DqW8RhOAW1k6xGU7pg61yzNvSBleX7STd5bsYuGWZJ7oEcftTbNpMKdCXlbnFjYfPkndSsX1LEMOtATkVk7cJRSkmApGhvO3brWZPagttSsWY/i0jdz+zgq2HTkVkOdTKlTpIrAbZLcbyIm7hIIckzGGz9Yc4IXZWzh1LoX721ZjcCdnNJhTwZXVVb5e+adxXDM4lUtXq6tHN3fOwJ8hyDGJCLc3jaZznQq8OGcL7yTsZtb6wzzbtx4d45zVYE4pp9EZgNN58ERwfqzc/RMjvkhiZ/JputeryNO961KphJ0Gcyr39Eo9sHQbqFsFuq7uxK2k+dCiehlmD2rL37vVZvG2ZDqPSeC9pXtISb1kOzSlHEdnAG4QqLq6E7eS+tEPP51h5IwkErYfI75KcUb3rU+D6JK2w1KX0cZzwaEzADeLbp5W9vH34Gz5JuyBFlOmMO/f24w3+jcm+eR5+r65jKdmJHHy3EXboSnlCLoI7GUOuAl7oIkIN15Xiba1yjJ23nYmr9jL3KQjPHVTXW6sXwkRd50dCLVaudNucuM1OgPwMgfchD1YiheMZFTvesx4pA3li0fx6JTvuWfSd/zw0xnboSllja4BKM9JvWT4YMVexszbzsXUSwzqVJM/t61OgYi8XQ8F4+pVa+X+56VZh64BKJUuPEy4t001FgxtT8e48vz76230HJ/Iyt0/2Q5NqaDSGYDyvEVbj/LUjE0c+Pkstzapyj971qF0kQI5fp2Nq/JgX7WG4lWyF2dTehLYi5zYKsKBOsZVoFX1soxftIP/LtnNgi1H+WePOtzWtGqWi8TaXvj/hGKC8BKdAYSqEN/jHyjbjpxixPSNrN73M81jSzO6Xzw1KxS74nMyD3qhOAjm9irZzT+7m2P3lc4AvMaJ7aJdoHbFYnz6YCs+W7OfF+Zspef4RAa2q86jN9Tknklpp6Uztxj2oqxaMIM3BtNQogkgVHlgj3+ghIUJdzSLoXOdCjw/eytvLN7FzPWHKBwZTsnCv18bcOOgl9OAHcqzmwyh+DP5ymoCEJGSwEQgHjDAfcYY715W+VPGHn9dA8izMkWjGHN7A25tUpUnv9jItqOnubF+JRpFl6RARJinBxAnJIhQTk7BYnsGMA6Ya4y5VUQKAIUtxxNanNgu2oVaXVuG2YPbMiFhN/9ZvJOU1EtElypM6iVDuMvuQuZr6UYH19BmLQGISHGgHXAPgDHmAnDBVjxKXU1URDh/7VSTmxpUZuSMJBJ3/Ei/N5fxfL/6xFcpYTs8a2xe+ev6Q/7ZnAFUB44Bk0SkAbAGGGyM+fXyTxKRgcBAgJiYmKAHqdTlYssW4YP7mjNrw2GembWZ3q8v5U+tYhnWtRbFCkbaDi9HTijdKOewtg1URJoC3wJtjDErRWQccNIYMzK7r9FtoC4T4ucQTpy9yCtfb+PDlfsoXyyKp2+qR4/4iq5oMOevBKBrAO7gxG2gB4ADxpiV6R9PBYZbjCf0BXNA9sA5hBKFInm2bzy3NKnKP6dt5OGP1nJD7XI80yee6NLOXs7SQVOBDwlARLoAtwNvGGPWichAY8yEvD6xMeaIiOwXkdrGmG1AJ2BzXr+fysH+VfD+jZB6EcIj4Z6vAjsge+gcQsPoksx8tA2TV+xj7LxtdHk14bcGc5Hhodluywl1eE1i+efLX+fDwN+BP4pIR6ChH57/r8BHIrIh/fs974fvqbKyfkraQIxJe7t+SmCfL9C3snSYiPAw7r++GguGtadDrfK8PHcbN45P5Lu9x22HplS2fCkBHTPG/AL8TUReBJrl98mNMeuA39WlVCBkrksHuE7t0XMIlUoU4u27m7BwS1qDudveXsEdTaMZ3iOOUrloMOcWupgcGnyZAXyV8Y4xZjjwgf/DUQHT4C4IjwIk7W2DuwL/nIG6laULdKpTgflD2/Fg++pMXXuATmMTmLrmAG7qvaVCX467gETkNWCIccBfru4CyqcQ35XjVFvXLWfE1wdZ83NhWlQrzeh+9alRvqjtsJSH5OeGMKeBmSJSOP0bdRWRZf4OUAWBh6/Irdm/irhZN/PZ2Yd4IWoyWw/9TI9xSxgzbxvnLqbajk55XI4JwBjzJPAxkCAiS4Fh6HZNZ9i/ChLHpL1VzpS+GyqMFO4KW8DCttu56brK/GfRTrq9toQl24/ZjjAk3fHOCk93a82tHBOAiHQC/gz8CpQDBhljEgMdmMpBxj77RaPT3moScKZMu6HK1m7N2DsaMuWBFoSL8Kf3VvHolLUknzxnO1LlQbnZBTQCGGmMWSoi9YFPRGSoMWZRgGNTV+Ohffauls1uqNY1yjLnsba8/c1u3vhmJwnbjvF499r0b3GN6xrMOYkTzie4SW5KQB2NMUvT398I9ACeC3RgKgce22fvatmsvURFhDO4c02+fqwd10WXYOSMTdz81nKSDp6wFKjymjz1AhKRQsaYswGI56o8vQsoqx08uqsnOILwezbGMHP9IZ6dtZnjv17g3jbVGNKlFkWjbHdsdye98r+SX3sB2Rj8PS27vjra7z/wgtTTSETo07BK2inir7fy3rI9zN54mKdvqke3ehVc0WBOuU9oNioJNVnV+1VwBPl3X6JwJKP71efzv7SmRKFIHvpwDQ9MXs2Bn88E9HlDzScPttKr/1zQBOAGmev9hcro9s9gsbTW0jimFLP6RTGiTjLLdybTZewS3k7YxcXUS0F5fuUN1u4HkBe6BpCYNvjPHW63zbLX1h5s/LyXlZ4OSkWeLj+OBXsvULtCMZ6/OZ4m15QOThwqJOTnJLBygoydJGd/slsO8uL5AxsnqC8rPVUxR5hYbwMT7m7CqXMXueWtFTwxbQO/nNE7qKr80QTgNra3f/qzJq4nmbOXxevctV5F5g9tzwPXV+PT1QfoNCaBaWu1wZzKOy0BuZHNEoy/dsV44I5h+XaV13nToROMmJ7Euv2/0Kp6GZ7rF8+15bTBnMpadiUgTQDKd/5IQIlj0spIJjXtKrfjiLQyi8q11EuGKat+4OW5Wzl/8RIPdbiWhztcS8HIcNuhKYdx4j2BlVv54/xBRokjYwagJ5l9Fh4m3N3yGrrVq8Bzs7YwfuEOZq47yHN963N9zbK2w1MuoDMAJ9HdNSofEnccY+QXSez96Qx9GlbmyRvrUq5YlO2wlANoCcjptCau/ODcxVTe/GYXb3+zi6jIMP7RPY7+zWMIC+EGc9r2IWe6DdTp9LSv8oOCkeEM7VKL2YPbUq9ycZ78Iomb31rO5kMnbYemHEjXAJxCa+LeksvyV16vbmuUL8rHf27J9O8PMvqrLdz0+lLuaxPLY51rUSREGsxp6+f8s/6XICLhwGrgoDGml+14rMmmb7wKQUFsMHdz46p0jCvPS3O38t/EPXy14TCjeteja72Kfn8+5T7W1wBEZCjQFCieUwII6TUA5R252AKb+eq2RbW01g/5ubpdvfc4I6Ynse3oKbrUrcCo3vWoUrJQnr+fU+iVf84cuQYgIlWBG4GJNuNQAaSnfX/P0mnuprGlmTXoeob3iCNxxzE6j0lgwhJtMOdlVmcAIjIVeAEoBvxNZwAhRnc2ZS/AawA5Pv3xMzw9cxOLtiYTV7EYz99cn8Yxpfz6HMo5HDcDEJFeQLIxZk0OnzdQRFaLyOpjx44FKTrlF7qzKXs2Gsxd/vSlC/PugKa8/cfG/HLmIre8tZwR0zdy4sxFK/EoO6zNAETkBeBuIAUoCBQHphlj/pjd1+gMwGV0BuAKp8+nMHbedt5fvofSRQowslddejeorHchCyGOPggmIh3QElDwBPMErhdP+7r0Z046eIIR0zey/sAJrq9Rlmf7xlOtbBHbYSk/0ASg0gTyqtylA59fuXzWk3rJMGXlPl6eu43zqZd4uMO1/KXDtURFuLvBnNd3CjluDeByxphvPH0GIJgCVZf34o1isuLydY/wMOHuVrEsHNaernUr8NqCHfR4LZHlO3+0HVrA3PHOit8ShNc4IgGoIArUFkSXD3x+Y/uGPX5SvnhBXu/fmMn3NSflkqH/xJUM+WQdP54+bzs0n2QM7iv3HGflnuOeHuyzYv0ksAqyQJ041lYWaULsRHf7WuWYN6Qdry/ayTtLdrFoazLDe8RxR9NonxvMOa0Mo60kNAF4kz/6+Wf1PUNo4MuXQPx+wdoaS8HIcP7WrTZ9G1VmxPQknpi2kalrDjC6XzxxFYsHLY68yBjMvTi454YjFoFzSxeB/WT/Klj/MWCgQX9vD9Zu4ZDFZWMMn689yOivNnPyXAoPXF+NwZ1rUrhA9teSgWhr4aurJQAvJAe9I5g/hMIul/2r4P1ekJpey/3+I7jnK90O6nRZrbFY+D2KCLc2qUqnuPK8OGcr7yzZzawNh3mmTz061akQ9HhyK5QH9/zQBJBbDrkCy7eMgSRD6sXADyah8rsLpJwSpMPWWEoVKcBLt17HLU2qMmL6Ru6fvJpu9dIazFUqcWWDOaeXYZwWTzDpLqDcCpVdLhkDSYbwyCsHk0A0bwuV312g5GYLbcYaS8cR2SdQC433mlcrzVeD2vJ499okbE9rMPfu0j2kaIM5V9AZQG457ArsN76WVqKbwz2zsl4DCNSVulN/d06R2/LO1RaXLc6yCkSE8XCHGtx0XWVGzkji2Vmbmbb2AKP71adhdMnfPs8fV9pOnUW4lSaA3HLiLpe8/qPPbiAJVJ3Zib87J/FHgnTAGkF06cJMuqcZc5KO8K8vN9HvzWXc3fIa/lbvNMUPL9PX3oE0AfgiUNv78srf/+gDeaXutN+dk2QkyIxZWV44ZJYlIvSsX4m2NcsyZt52Plixlznf/sLIiIXcVODfyD15m5nonv3A0ATgZv7+R69X6nat+zjttVz3P99LOA577YoVjGRU73rcEraYfy4zDLr4CJ+lbuC5Td9yjf5dOYaeA3A73V4ZGnJxm0hX2r+K1Pf78OGFtvz74u1cDC/Eox1rMbB99Tw1mNMr/7zRcwChSksrocEhJRy/i25O+D0zGLA3ke7lqvDMmijGzN/OF+sOMrpffVpWL2M7Qk/TGYBSTuGR2dzibck8NSOJ/cfPckvjqoy4sQ6lixTI+QtVnjn6fgC5pQlAqVxwQSI5eyGV/yzawYQluylaMIInesRxWxPfG8yp3NEEoJQXuOzU9fajp3hyehKr9h6nWWwpRverT60KxWyHFXIcfUMYpQLCwslY61x26rpWhWJ88mBLXr71OnYmn6bnuERemruVsxdSbYfmCboIrEKTy66E/caFi8kiwu1No+lcpwLPz97CW9/sYtaGQzzTJ54bape3HV5I0xmACk0uuxL2m9z0DHKo0kUK8MptDfjfwJZERYRz76TvePijNRw5cc52aCFLZwAqNLnwSthvXL41uGX1Mswe1Jb/Ju5m/MIdLNn+I8O61uJPrWIJ99cisQsWyoNBF4FV6NJ/5K6376dfGTljE0u2H6N+lRKM7hfPdVVL5vyFV+PB8qAuAqs0XloYjW6edpo2xP9x58jFr/k1ZYow+d5mvN6/EUdOnqPvG8sYNXMTp85dzPs39Wp5MAvWSkAiEg18AFQELgETjDHjbMWTJ267wvTglY/nhcBrLiL0uq4y7WqVY8zX25i8Yi+zNx7m6Zvq0bN+RUR8LAt5uTyYic0ZQAowzBhTB2gJPCIidS3G45vc3MTDafTKx3tC6DUvXjCSf/WJ54uH21CuWBSPTFnLve9/xw8/nfHtG7l4odzfrCUAY8xhY8za9PdPAVuAKrbi8Zkb/2FlXPlIuOevfDwjUK+5xbJSg+iSzHikDU/1qst3e47T5dUE3li8kwspPtyFTMuDgEMWgUUkFlgCxBtjTmb6fwOBgQAxMTFN9u3bF/T4suTWqbXbylYq//z9mjvob//IiXP868tNzEk6Qs3yRRndrz7Nq5W2EouTObYVhIgUBRKA0caYaVf7XMftAgqVwTRUfg4VHA5sXb1o61FGfrGJg7+c5famVXmiRx1KaYO53ziyHbSIRAKfAx/lNFJkm/AAAAvPSURBVPg7ksv3WwOOuppTQZSfpO/ARdSOcRVoObQM4xfuZGLibuZvPso/e9bh1iZVfV8k9hCbu4AEeBfYYowZaysOz3PAvWRVkOU36Tvs7mMZCheIYHiPOPo2qsyT05P4+9QNTF1zgNH94qlRXhvMZcXmLqA2wN1ARxFZl/5fT4vxeJPTF4ZdvIfdsfyxgcHBi6hxFYvz6YOtePHm+mw9cooe4xJ55ettnLuoDeYyszYDMMYsBXRuZptDr+YALU8FigNLOP4WFibc2TyGLnUrMHr2Fl5fvJOZ6w/xbN942tcqZzs8x9CTwMq5V3Nu3GrrBh7aB1+maBRjb2/IlD+3ICJcGPDeKh6dspbkk9pgDjQB5J6WIoLPZnkq1F9vpyb9AGl9bVnmDG7L0C61mLf5KJ3GJPDBir2kXrK/Dd4m69tAfWFtG6iWIuyxsUVVX++QtvfHXxk5I4nEHT/SoGoJRverT3yVErbDCihtBpcfWoqwx8aVqr7eIS22bBE+uK854+9qxMFfztH79aU88+VmTp9PsR1a0GkCyA2n75RR/qWvd8gTEXo3qMzCYe3p3yKGScv30HlMAnOTDuOmqkh+aQkot/S0rH85/ffp9PiUX33/w8+MmJ7E5sMn6RhXnn/1rkd06cK2w/Ibx7aC8IXjWkGovNEau3KglNRLvL98L2Pnb+eSMQzuVIsH2lYjMtz9hRJdA1DOoTV25UAR4WE80LY6C4a2p13Ncrw0dyu9xi9l9d7jtkMLGE0AKm/ys01Sa+zKwSqXLMSEPzXlv39qyunzKdz69gqGf76BX85csB2a32kJSPnOHyUcrbErFzhzIYVxC3YwcekeShSKZETPOtzcuIrrGsxpCUj5jy8lnOxmCh47iKTcqXCBCJ7oWYdZf72e2DKFGfbZevr/dyW7jp22HZpfaAJQvsttCceNt81UKgt1KhVn6kOteb5ffTYdOkGP1xIZO8/9DeY0ASjf5baXjC72qhASFib0bxHDwmEduPG6SoxftJNury0hcccx26HlmSYAlTe5KeHoYq8KQeWKRfHqHQ2Z8kALwkS4+91VDPr4e5JPua/BnC4ChwqnLqo6NS6l/ODcxVTeTtjFm4t3ERUZxuPd4+jfPIbwMGctEutBsFCmB6vs0iTnebuPnWbkjCSW7fyJBtEleb5fPPUqO6fBnO4CCmVaa7dHF7oVUL1cUT68vwWv3dGQgz+f4ab/LOXZWZv51eEN5jQBhAKttdujyVelExH6NqrCwqEduLN5DO8u3UPnsQl8vemI7dCypQkgFHjoDk+Oo8lXZVKicCTP96vP539pTYlCkTz4/9bwwOTVHPj5jO3QfkfXAJTKL10DUNm4mHqJScv28Or8HQA81rkm910f/AZzugislFKWHPzlLE/P2MSCLUeJq1iM0f3q0+SaUkF7fl0EVsqf8tIML9TvM6yyVaVkISYOaMo7dzfhxNmL3PLWcp6YtpETZy5ajSvC5pOLSHdgHBAOTDTGvGgzHqVyJS/bbnWrrgK61avI9TXK8ur87Uxavpd5m47wZK869G1op8GctRmAiIQDbwA9gLrAXSJS11Y8ymGcfLWcl50/ultIpSsSFcGTveoy89E2RJcuzJBP1vOHiSvZbaHBnM0SUHNgpzFmtzHmAvA/oI/FeJRTOH1vfV52/uhuoZw5OekHQL3KJfj8L615tm88Gw+eoPtribw6f3tQG8zZLAFVAfZf9vEBoEXmTxKRgcBAgJiYmOBEpuzK6mrZSeWSjG23vuz8ycvXeIlHS2ThYcLdLa+hW70KPDdrC+MW7mDm+kM82yee62uWDfgOM5sJIKuC1++2JBljJgATIG0XUKCDUg6QcbWcMRg48Wo5urnv/yDz8jVe4fSkH2DlixVk/F2NuLVJVUbOSOKP766kb60oRhx4mHLmx4AlRZsloANA9GUfVwUOWYpFOYkebPMeLZEB0K5WOb5+rB2DOtbgqx3nmHuxQUDXjWzOAL4DaopINeAgcCfQ32I8oc9NB5b0atlbtET2m4KR4QztWpubKx0n+oulcClwSdFaAjDGpIjIo8DXpG0Dfc8Ys8lWPCEv1GqsbkpmKnc06V8htn4rKDkjZNcAMMbMBmbbjMEzQqnGGmrJTKnsBDgp6klgrwilGqvuqVfKL6zOAFQQhVKN1Q27hJRyAU0AXhIqNVZ/JrNQXUsI1Z9L+ZUmAOVO/khmobqWEKo/l/I7XQNQ3hWqawnB+Lk81rYhVOkMQHlXqK4lBPrn0hlGyNAEoLwrlBbGLxfonyuUthR7nCYA5W2hsjCeWSB/rlCdOXmQJgCllG9CdebkQZoAlFK+C9WZk8foLiCllPIoTQBKKeVRmgCUUsqjNAEopZRHaQJQSimP0gSglFIepQlAKaU8SowxtmPINRE5BuzL45eXBX70YzjB4ta4wb2xuzVucG/sGndgXWOMKZf5QVclgPwQkdXGmKa24/CVW+MG98bu1rjBvbFr3HZoCUgppTxKE4BSSnmUlxLABNsB5JFb4wb3xu7WuMG9sWvcFnhmDUAppdSVvDQDUEopdRlNAEop5VGeSgAi8m8R2SoiG0RkuoiUtB3T1YhIdxHZJiI7RWS47XhyQ0SiRWSxiGwRkU0iMth2TL4SkXAR+V5EZtmOJbdEpKSITE3/+94iIq1sx5RbIjIk/W8lSUQ+FpGCtmPKioi8JyLJIpJ02WOlRWS+iOxIf1vKZoy+8lQCAOYD8caY64DtwBOW48mWiIQDbwA9gLrAXSJS125UuZICDDPG1AFaAo+4JO7LDQa22A7CR+OAucaYOKABLolfRKoAg4Cmxph4IBy4025U2Xof6J7pseHAQmNMTWBh+seu4akEYIyZZ4xJSf/wW6CqzXhy0BzYaYzZbYy5APwP6GM5phwZYw4bY9amv3+KtIGoit2ock9EqgI3AhNtx5JbIlIcaAe8C2CMuWCM+cVuVD6JAAqJSARQGDhkOZ4sGWOWAMczPdwHmJz+/mSgb1CDyidPJYBM7gPm2A7iKqoA+y/7+AAuGkgBRCQWaASstBuJT14DHgcu2Q7EB9WBY8Ck9NLVRBEpYjuo3DDGHAReAX4ADgMnjDHz7EblkwrGmMOQdvEDlLccj09CLgGIyIL0WmLm//pc9jkjSCtVfGQv0hxJFo+5Zs+uiBQFPgceM8actB1PbohILyDZGLPGdiw+igAaA28ZYxoBv+KSUkR6zbwPUA2oDBQRkT/ajco7Qu6m8MaYzlf7/yIyAOgFdDLOPgRxAIi+7OOqOHRqnJmIRJI2+H9kjJlmOx4ftAF6i0hPoCBQXEQ+NMY4fUA6ABwwxmTMtKbikgQAdAb2GGOOAYjINKA18KHVqHLvqIhUMsYcFpFKQLLtgHwRcjOAqxGR7sA/gN7GmDO248nBd0BNEakmIgVIWxibaTmmHImIkFaL3mKMGWs7Hl8YY54wxlQ1xsSS9vte5ILBH2PMEWC/iNROf6gTsNliSL74AWgpIoXT/3Y64ZIF7HQzgQHp7w8AZliMxWchNwPIwetAFDA/7W+Nb40xD9kNKWvGmBQReRT4mrSdEe8ZYzZZDis32gB3AxtFZF36Y/80xsy2GJMX/BX4KP1iYTdwr+V4csUYs1JEpgJrSSvLfo9D2yuIyMdAB6CsiBwAngZeBD4VkftJS2a32YvQd9oKQimlPMpTJSCllFL/RxOAUkp5lCYApZTyKE0ASinlUZoAlFLKozQBKKWUR2kCUCoP0lted0l//zkRGW87JqV85bWDYEr5y9PAMyJSnrSGd70tx6OUz/QgmFJ5JCIJQFGggzHmlIhUB0YAJYwxt9qNTqmcaQlIqTwQkfpAJeB8+n0PSL93w/12I1Mq9zQBKOWj9K6PH5HWxvhXEelmOSSl8kQTgFI+EJHCwDTSbnu5BXgWGGU1KKXySNcAlPITESkDjAa6ABONMS9YDkmpq9IEoJRSHqUlIKWU8ihNAEop5VGaAJRSyqM0ASillEdpAlBKKY/SBKCUUh6lCUAppTxKE4BSSnmUJgCllPKo/w/atXXbN39DbAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "def sigmoid(z):\n",
    "    return 1 / (1 + np.e**(-z))\n",
    "\n",
    "def draw_samples(X, Y, sample_count):\n",
    "    ''' 绘制正负例.\n",
    "        将正负例的X1 X2标签分别识别出来，并最终绘制\n",
    "    '''\n",
    "    positiveX1 = []\n",
    "    positiveX2 = []\n",
    "\n",
    "    negativeX1 = []\n",
    "    negativeX2 = []\n",
    "\n",
    "    for i in range(sample_count):\n",
    "        if Y[i, 0] == 1:\n",
    "            positiveX1.append(X[i, 0])\n",
    "            positiveX2.append(X[i, 1])\n",
    "        else:\n",
    "            negativeX1.append(X[i, 0])\n",
    "            negativeX2.append(X[i, 1])\n",
    "    \n",
    "    plt.scatter(positiveX1, positiveX2, marker='+')\n",
    "    plt.scatter(negativeX1, negativeX2, marker='.')\n",
    "\n",
    "def draw_border(theta):\n",
    "    '''绘制边界线'''\n",
    "    X = []\n",
    "    Y = []\n",
    "    for x in range(-2, 12):\n",
    "        X.append(x)\n",
    "        # theta[0] + theta[1]x1 + theta[2]x2 = 0\n",
    "        Y.append(-theta[0] / theta[2] - theta[1] / theta[2] * x )\n",
    "    plt.plot(X, Y)\n",
    "\n",
    "\n",
    "def create_samples(samples_count):\n",
    "    ''' 生成样本数据\n",
    "        以x1+x2-10 = 0为界限  生成100条数据，大于0标记为1   小于0标记为0\n",
    "        并添加随机噪声  \n",
    "    '''\n",
    "    X = np.empty((samples_count, 2))\n",
    "    Y = np.empty((samples_count, 1))\n",
    "\n",
    "    for i in range(samples_count):\n",
    "        x1 = np.random.randint(0, 10)\n",
    "        x2 = np.random.randint(0, 10)\n",
    "        y = 0\n",
    "        if x1 + x2 - 10 > 0:\n",
    "            y =1\n",
    "        X[i, 0] = x1\n",
    "        X[i, 1] = x2\n",
    "        Y[i, 0] = y\n",
    "\n",
    "        noise = np.random.normal(0, 0.1, (samples_count, 2))\n",
    "        X += noise\n",
    "    return X, Y\n",
    "\n",
    "def grad(X, Y, samples_count, theta):\n",
    "    ''' 求代价函数在theta处的梯度. '''\n",
    "    #求出预测值\n",
    "    T = sigmoid(np.dot(X, theta))\n",
    "    #求出梯度  这里似乎不需要除以样本数,因为下面会根据梯度的范式决定停止条件，所以这里不做修改\n",
    "    g =  np.dot(X.T, (T - Y)) / samples_count\n",
    "    return g\n",
    "\n",
    "def descend(X, Y, samples_count, theta = np.array([[1],[1],[1]]), step = 0.01, threshold = 0.05):\n",
    "    ''' 梯度下降.\n",
    "        Args: \n",
    "            X: 样本\n",
    "            Y：样本标记\n",
    "            samples_count：样本个数\n",
    "            theta：初始值\n",
    "            step：步长\n",
    "            threshold：阈值\n",
    "        Returns:\n",
    "            theta：求出来的最优theta\n",
    "    '''\n",
    "    g = grad(X, Y, samples_count, theta)\n",
    "    norm = np.linalg.norm(g)\n",
    "\n",
    "    while norm > threshold:\n",
    "        g = grad(X, Y, samples_count, theta)\n",
    "        norm = np.linalg.norm(g)\n",
    "        theta = theta - g * step\n",
    "        #查看每次梯度向量的L2范式\n",
    "        #print(norm)\n",
    "\n",
    "    return theta\n",
    "\n",
    "samples_count = 100\n",
    "X, Y = create_samples(samples_count)\n",
    "MatrixOnes = np.ones((100, 1))\n",
    "X_with_noes = np.hstack((MatrixOnes, X)) # 添加等于1的x0\n",
    "theta = descend(X_with_noes, Y, samples_count)\n",
    "\n",
    "plt.xlabel('$x_1$')\n",
    "plt.ylabel('$x_2$')\n",
    "\n",
    "draw_samples(X, Y, samples_count)\n",
    "draw_border(theta.flatten())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## sklearn 案例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- penalty 正则化  \n",
    "        penalty='l2' 默认是L2正则化。如果是L2正则化，则优化方法可选{‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’}\n",
    "        penalty='l1' 如果是L1正则化，则优化方法可选{‘liblinear’} 其他方法都需要求一阶或二阶导数\n",
    "        liblinear：坐标轴下降法\n",
    "        lbfgs：拟牛顿法的一种，利用损失函数二阶导数矩阵即海森矩阵来迭代优化损失函数。\n",
    "        newton-cg：也是牛顿法家族的一种，利用损失函数二阶导数矩阵即海森矩阵来迭代优化损失函数。\n",
    "        sag：即随机平均梯度下降，是梯度下降法的变种，和普通梯度下降法的区别是每次迭代仅仅用一部分的样本来计算梯度，适合于样本数据多的时候。\n",
    "- multi_class\n",
    "        one-vs-rest(OvR)    K次 次   四种损失函数优化方法都可选\n",
    "        many-vs-many(MvM)   K(K-1)/2 次 速度慢  不能选liblinear\n",
    "- class_weight\n",
    "        dict ：class_weight={0:0.9, 1:0.1} 自定义某类别权重\n",
    "        balanced：根据样本比例自动计算\n",
    "        好处：\n",
    "        1.如果误分类的代价很高，如对非法用户提高权重  \n",
    "        2.类别不平衡 对少类别的样本提高权重\n",
    "- sample_weight\n",
    "        在fit的时候可以用sample_weight调节每个样本权重"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
    "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
    "                   multi_class='warn', n_jobs=None, penalty='l2',\n",
    "                   random_state=None, solver='lbfgs', tol=0.0001, verbose=0,\n",
    "                   warm_start=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAELCAYAAADURYGZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hUZdrH8e+d0HsLvYSOlIAQQboiHQRBXbus7gp2BF3FBV+74O6KIDYQXXXtS5EiXRQCUgSEEDrSmwSRonTyvH/MjBtDymQyc8qc+3NdXiEzk8w9k/Hc53mec35HjDEopZTyrhi7C1BKKWUvbQRKKeVx2giUUsrjtBEopZTHaSNQSimP00aglFIeZ1kjEJH3ROSwiKRkct9jImJEpJxV9SillPKxckTwPtA9440iUg3oAuyxsBallFJ+ljUCY8xi4Ggmd70KPA7omW1KKWWDfHY+uYj0AfYbY9aJSE6PHQgMBChatGiLBg0aWFChUkpFj9WrVx8xxsRlvN22RiAiRYDhQNdgHm+MmQBMAEhMTDSrVq2KYHVKKRV9RGR3ZrfbedRQbaAmsE5EdgFVgTUiUtHGmpRSynNsGxEYY9YD5QPf+5tBojHmiF01KaWUF1l5+OinwDKgvojsE5G/WPXcSimlsmbZiMAYc0sO98dbVIpSSql09MxipZTyOG0ESinlcZ5qBMt3/MzOI7/ZXYZSSjmKZxpBWpphxJcpdB+zmLcX/ciFi2l2l6SUUo7gmUYQEyN8/NdWdKwXx6jZm7nuzaVsPHDC7rKUUsp2nmkEABVKFGL8HS1487bmHDp+hj6vL+Gfczdz5vxFu0tTSinbeKoRAIgIPZtUYsHQjlx3eRXe+OZHer6WxKpdmeXhKaVU9PNcIwgoVaQA/7qxKR/e3ZKz59O4cfwynp6Wwq9nL9hdmlJKWcqzjSCgQ7045g3pwIDW8Xy4fDddRy/imy2H7S5LKaUs4/lGAFC0YD6e6dOISfe2oUjBfNz17+8Z+vlafvntnN2lKaVUxGkjSKdFjdJ89XA7Hu5Uh+nrDtB59CJmrDuAMXrNHKVU9NJGkEHBfLEM7VqfGQ+1o0rpwjz06Q/c8+FqDh0/Y3dpSikVEdoIsnBZpRJMua8Nw3texpLtqXQZvYhPVuwhLU1HB0qp6KKNIBv5YmO4p0Mt5j7SgcZVSvL3qeu5deJydmlMhVIqimgjCEKNskX55J5WjOrfhA0HTtBtzGLGa0yFUipKaCMIkohwc8vqLBjakQ714hg5ezP93vxOYyqUUq6njSCXKpQoxIQ7WvDGrc05ePw0fV5fwivztnD2gsZUKKXcSRtBCESEXgmVmD+kI32bVWHcwu30HKsxFUopd7LymsXvichhEUlJd9s/RWSziCSLyFQRKWVVPeFQumgBXvlTUz64uyVn/DEVz0zfwG8aU6GUchErRwTvA90z3DYfaGyMSQC2Ak9aWE/YdEwXU/HBsl10fXUx32pMhVLKJSxrBMaYxcDRDLfNM8YEdp+XA1Wtqifc/hdT0ZpC+WP487+/Z+gXGlOhlHI+J60R3A3MzupOERkoIqtEZFVqaqqFZeVOixplmDW4vS+mYu0Bury6iJnJGlOhlHIuRzQCERkOXAA+zuoxxpgJxphEY0xiXFycdcWFIH1MReVShXnwkx8Y+J/V/HRCYyqUUs5jeyMQkQFAb+A2E2W7zeljKpK2pdJ59CI+XblHRwdKKUextRGISHfgCaCPMeaUnbVESiCmYs7gDjSqXIInp6zn1ndWaEyFUsoxrDx89FNgGVBfRPaJyF+A14HiwHwRWSsib1tVj9XiyxXl03uuZGT/JqTsP073sYuZsFhjKpRS9hM3TlMkJiaaVatW2V1GyA4dP8NT01KYv/EnEqqW5OXrE7isUgm7y1JKRTkRWW2MScx4u+1rBF5UseT/YioOHDvNteM0pkIpZR9tBDZJH1PRp1llxi3cTq/XlrB6t8ZUKKWspY3AZqWLFmD0n5rxwd0tOX3uIje8rTEVSilraSNwiI714pibIaZi0VbnnjinlIoe2ggcpFiGmIoB763UmAqlVMRpI3CgFjXK8NXD7XkoXUzFV8kH9UQ0pVREaCNwqEL5Y3k0XUzFA5+s0ZgKpVREaCNwuEBMxd97NmDxVl9MxWcaU6Ec5qbxy7hp/DK7y1Ah0kbgAvliYxjYoTZzH/HFVAzzx1Ts/lljKpRSeadnFruMMYbPvt/LS19t4nxaGo92qc9dbePJF6s9XVkvMApYsdN3/kurmmUA+HxQa9tqUlnTM4ujhIhwS8vqzB/akXZ14nhx1ib6v/Udmw6esLs0pZRL6YjAxYwxfLX+IE9P28Dx0+e5/6raPNCpDgXzxdpdmvKYwMhARwLOpiOCKCQi9E6ozIKhvpiK1zSmQikVAh0RRJFvtxxm+NQUDhw/zYDW8fytW32KFsxnd1lKKYfQEYEHXFW/PHOHdODOK2v8HlOxWGMqlFI50EYQZYoVzMezfRvz30G+mIo731vJo1+s49gpjalQSmVOG0GUSoz3xVQ8eHUdpq3dT+fRGlOhlMqcNoIoVih/LI91q8/0B9tRqaQvpmKQxlQopTKw8prF74nIYRFJSXdbGRGZLyLb/F9LW1WPlzSsXIKp97fhyR4NWKQxFa6i0Q3KClaOCN4Hume4bRjwtTGmLvC1/3sVAfliYxjU0RdT0bCSL6bitokaU6GUsvjwURGJB2YaYxr7v98CXGWMOSgilYBvjTH1c/o9evho3qSl+WIqRs76X0zF3e1qEhsjdpem/DS6QUWCUw8frWCMOQjg/1o+qweKyEARWSUiq1JTo+CQyL0rIekV31eLxcQIt7YKxFSU88VUvLmUzYc0pkIpL7J7RHDMGFMq3f2/GGNyXCdw/Yhg70r4oA9cPAexBWDAdKjW0pZSjDHMTD7IM9M1psKJNLoh/Lz8njp1RPCTf0oI/9fDNtdjjV1JviZgLvq+7kqyrRQR4dqm/piKpuljKn4J7RfaONJRSoXG7vyB6cAAYJT/6zR7y7FIfHvfSCAwIohvb3dFlC5agNE3NaNPs8oMn5rCDW9/l/uYCgeNdKKFF/daIyXjuouXRwYZWXn46KfAMqC+iOwTkb/gawBdRGQb0MX/ffSr1tK3kew03HEbyzzFVDhopKOUCp6Gzqksrdp1lMcnJ7Mj9Teub16Vp3pfRqkiBbL+AQeOCHSvT2Xk5c9EVmsEdk8NKQdLjC/DrIfb8/rC7by96EcWbT3Mc30b06NxRUQyOdQ0MNLZleSb7nLQSEcplTUdEaigbDhwnCcmJ5Oy/wRdG1bg+esaU6FEIbvLypIeh6/UpZx61JByiUaVS/Ll/W01piLCNFJC2UGnhlTQAjEVXRtVZNjkZIZNWc/0dQcY2b8JNcoWtbu8Pwjs+Xt5PlipYGkjULlWs1xRPr3nyt9jKrqNWawxFXmkhzYqO2kjUCEJxFRc3SCOp75M4cVZm5iZfICXb0igQcUSdpf3O92QKpUzXSxWeXZJTMXVdXjg6toaUxECK0cCThx1OLGmaKKLxSpiAjEV84d25NqmlXnt6215i6lQSllKRwQq7L7ZcpjhU9Zz8MQZ/twmnse6BhdToXuD1nDiobVOrCka6YhAWebq+uWZN7Qjd15Zg38vzWVMhVLKcjoiUBH1/a6jPOGPqbihRVVG9Lo0pkL3BoMXzlGTE0dgTqwpmuiIQNniCn9MxYNX12HqD/vpPHoxs9Yf1BPRlHIQHREoy6SPqejWqALP921M+XQxFbo3mDUdNeWNfrZ8dETgJlF6cZf0MRXfbknlmtGL+Px778ZUaJyEcgodETiNA6OcI2Hnkd8YNjmZFTuP0qZ2WUb1T6B62SJ2l2WpUPZSdc82d3Qk9UcaQ+0WWV3cJato570rXRn7HIip+PT7PYyctZmuYxbxWNf63NU2+mMqNE5COY02AqfJeBnLwmWzHiG4fPQQEyPc1qoGnRqUZ8TUFF74ahMzkg/yj+sTqF+xuN3lOZI2i9zR8MHgaCNwmowXd8lshBDY2Gd3n4tUKlmYiQMSmeGPqeg9Lon7rvpjTEU0/Y+sGyflNI5oBCIyBPgrYID1wF3GmDP2VmWjai3/uEHP6kL3GUcP6e9zGRGhT9PKtKtTjudmbOC1r7cxe/1BXr4hgebVS9tdnnK5SDXbaGnmti8Wi0gVYAnQ0BhzWkS+AGYZY97P6meierE4M9mtA7h0jSAn32w+zPCp6zlw/AwVSxTk0ImzgC72KWdxWyNw+mJxPqCwiJwHigAHbK7HWTKOEIK9z8WublCeuUM6cM0ri35vAtHAbRsOlbloW/C3/TwCY8x+4F/AHuAgcNwYMy/j40RkoIisEpFVqamaW+MFxQvlZ+XwznwxqDWF8vs+qtXLFGHCHZfs0KhsROJ8BT0HIrrYPiIQkdJAX6AmcAz4r4jcboz5KP3jjDETgAngmxqyvFBlm5Y1y9Ckckn2HzvNlB/2882WVJ7v24geTSrZXVrQom0P0uuibcHf9kYAdAZ2GmNSAURkCtAG+Cjbn1Ke8t/72gCQst8XU3Hfx2vo3qgiz/Vt9IeYCvU/kWg+0dDQ3FhzpDmhEewBrhSRIsBp4BrAQyvBKjcaVynJtAfa8k7STl5dsJXvRh9hRK+G3JhYFRHnnogWbXuQyida/o62HzUEICLPAjcBF4AfgL8aY7JcIfTcUUMqUztSf2XYlPWs3HmUdnXK8VK/Jo6PqbCjEUTiOd0Yh61xEw4/asgY8zTwtN11KHepFVeMz+65kk9W7mHU7M10G7OYR7vWc3RMhZc2OuGgIyhrOGJEkFs6IlAZHTh2mhFfprBw82GaVivlqZgKt24sg9lDd/poxm0cPSJQKq8qlyrMuwMSmb7uAM/O2EjvcUncf1Ud7k8XU6HcI5IL3epS2ghU1BAR+jar4oupmLmRsV9vY3bKQV6+PoHLozCmwu1H8Ni1gO6W98dK2giUtSyIxChbrCBjb76cvs0qM3xqCv3f+o672tTksW71KFIg64+82zak0SycTcLtDdMKtp9ZrDwkEJu98EXf1whfga1TgwrMG9KB21pV572lO+n66mKWbDsS0ee00ueDWvP5oNa0qlmGVjXLXLLxdIvA61D20RGBso6Vsdn+kUfx+Pa8cF1Lrk2ozLAp67n93RXc2KIqI3o1pGSR/IDuMTpZOP4Geg5HzrQRKOsULgsiYGIiG5udyQV7WtVqyezB7Rn79TYmLN7Bt1tTea5tQXrErqTuuVJsK9AwMrVY4PNBrX/P/tFmpkKhjUBZY+9KmDMMTBrExED3UZEbDWQx8iiUP5YnujegV5NKPP7pcu6bc4LuMSd5rtAEyt/1H26a5YwTjHQjHhn6fmZNG4Gyxu8b5zRA4PTPkXuuHC7Y07hKSaZdkcI789cx5kI/vjvVkBFJyRjT0NExFdnR6Q+VF9oIlDWsvJpaxst9ZjLyyF+rPfcX+ifdzq/iyfP38HhyPdrVieWlfk0iV1cOdK1C2UUbgbJGEBvnsD9fds/hr6f2riQ+q9GGjw9U5GWXxFRkR5uGCoVGTCjllz6molm1UrxsU0yFjgRUpGQVMaHnESjlF4ipGHtzM3b//Bu9xyXx6vytnLuQZndpSkWUjghU3lhwprAdfv71LM/N3Mi0tQeoV6GY62IqdFShMqMjAhV+Fp8pbKVATMW7AxI5cfoC/d/6judnbuTUuQt2l6ZU2OlisQqdlWcK2+SayyrQsmYZXp6zmXeX7GTexkOM7JdAu7rl7C4tU3rkkQqFjghU6AKHhEps5A8JtVHxQvl54bomfD7wSvLFxHD7uyt4fNI6jp86b3dpSoWFrhFEu0jP4UfpGkFWzpy/yJgF23gnaQdlihbg+b6N6N64kt1lXUJHAiozeV4jEJEuIvKOiDTzfz8wjMWVEpFJIrJZRDaJiH56w8GKOfxqLaH9o55oAgCF8scyrEcDpj3QlrhiBbn3ozXc99FqDp88Y3dpSoUsN2sE9wN3ASNEpAzQLIx1jAXmGGNuEJECgLOvQO4WHpjDD7B6D7hxlZJMe7At7yTtYMyCbSzdfoSnejfkhhZVwxZTkZfXpCOB7OmI6Y9ys0aQaow5Zox5DOgKXBGOAkSkBNABeBfAGHPOGHMsHL/b8zwyh2+X/LEx3H9VHWYPbk/9isX526Rk7nxvJXuPnrK7NKVyJeg1AhHpa4yZlu77h4wx4/JcgG+qaQKwEWgKrAYGG2N+y/C4gcBAgOrVq7fYvXt3Xp/aG6J8Dn/EuHdpdC6Z/x6JZ42pl+kF0K2Qlmb4eOUeRs3aRJqBv3Wrz4A28SHFVARzUXcVGq+/tyGvEYjIGBGR9E0AIBxNwC8f0Bx4yxhzOfAbMCzjg4wxE4wxicaYxLi4uDA9tQdE8xz+3pU89fOT/Onkh3xc4CWay1bbSomJEe64sgbzhnbkylpleG7mRq5/6zu2/nTStpqUClaOIwIReQHfnvpNxphTItIVeNoY0zYsBYhUBJYbY+L937cHhhljemX1M3rUkAIg6RXfQri5yEVi+KL4ndzy6Fi7q8IYw/R1B3hm+gZ+PXuBB6+uy31X1aZAvtwdra3z2JFz0/hlbDx4goaVSnjq/Q15RGCMGQF8CiwSkSXAo2Syxx4qY8whYK+I1PffdA2+aSKlspduDeQC+dlQIMHuigAQEfo2q8KCoR3p0bgSry7YyrXjlrB2ry59KWcKZkRwDTACEKAS0McYsyWsRfjWCSYCBYAdwF3GmF+yeryOCCLMTesKLqj1600/MXxqCodPnuHutjUZ2rUeRQroSf128fI6QVYjgmA+jcOBp4wxS0SkCfC5iAw1xiwMV3HGmLXAJcUpG2RyvV+nbmCBnK874ACBmIpRszczcclO5m48xKj+CbSt48yYCuU9wUwNdTLGLPH/ez3QA3gh0oUpm2R27oHKs+KF8vNivyZ85o+puG3iCp6YlKwxFcoRcp01ZIw5iG8eX0UjPfcgoq6sVZbZg9tzb8faTFqzjy6vLmJOyiG7y1Iep1lD6lIumHePBin7j/P4pGQ2HjxBzyYVeaZPI8oXL5T5g/VvEnZePCorL2sEymtcMO8eDQIxFRMW72Ds19tYuv1nRvS67NKYChet23hx4xoNNIZaKRvlj43hgat9MRX1KhTLPKZC120i4vNBrbVh+emIQDlbsFMiLp86qR1XjM8HtubjFbsZNXsz3cYs5rGu/piKwLpNYETgwHUbvSCOu2kjUKGz4loHwUyJuGjqJDsxMcIdrePpdFkFhk9dz3MzNzIj+QD/uD6BugOmu7rRKWfTRqBCY8XGN9gY7SiL265SqjD//vMVTFt7gGdnbKDXa0t44Oo63HfVkFzHVFglsOevIwF3cuanSjmfFfPW8e0hJhYQ39espkSi8JBXEeG6y30xFd0bV/w9pmKdxlSoCNARgQqNZfPWkuFrJqq19I1IonDqpGyxgrx2y+X0aVqZEV+m0O/NpfylXU2GdqlP4QKxdpd3CR0JuJM2AhUaKza+u5Ig7QJgfF+zm/KJ8kNeOzesQMtaZXh59mbeSdrJ3A0/Map/E9oEGVOhUzYqOzo15EV7V/oinHN7DeOMPxfpax1E4ZRP0DL5G5VIF1MRGyPcOnEFwyYnc/y0xlTk1k3jl/3eHNP/26t0RBDtMh7ZE+oirx1H5kTxlE+2cnivAzEVYxZs452kHSzcfJjnr2tMt0YVL/lVelinCoY2gmiW2QYl1CNs7DoyJ8qnfDIVxHtdKH8sw3o0oHdCJR6flMyg/6ymV5NKPNOnEXHFC9pUeNac0oAyNsYmz8zl5JkLf7jP7hrtoI0gmmW2QQl1kdcFJzX9gZtPMMvkvc5qI5UxpmLJ9iM81bsh1zevgoi48rBON9UaLbQRRLPMNt6hTre4aZrG7SeYZfpeZz2HHYip6NaoIsMmJ/PYf9cxbe1+XurXhGplilhXdyacNjWVWWO0uyYn0EYQzbLaeIc63eKWaZpoOMHM/177NlLLgtqQ1ilfjC8G/TGm4m/d6nNn63hXbOSc1jS8RBtBtHPLxjuc3DaNFUYZYyqenbGRGesO8PL1CdStUNzyeuycmsruOdPfpo3GQY1ARGKBVcB+Y0xvu+tRDpbT/L+bprFyEOqGNLOYigc71eHejrU1pkJdwjGNABgMbAJK2F2IcrBg5/+9OBLKIBBT0a5uOZ6dsZHR87cya/1BXr4+gabVSllaix0jAZ1iCp4jdg1EpCrQC5hody3K4TyazZ+X7PxyxQoy7pbLmXhnIsdOnaffm0t58auNnD53McxVZi/YE7f0OgHWc8qIYAzwOJDlJKaIDAQGAlSvXt2ispTjOHH+3yWHqgZiKkaFGFPhFjrFlHu2NwIR6Q0cNsasFpGrsnqcMWYCMAF81yy2qDzlNE6b/3fZoaolCuXnpX5N6NO0MsMmJ3PrxBXcfEU1nux5GSUL58/17wtmY+vEqRon1OAktjcCoC3QR0R6AoWAEiLykTHmdpvrUk7lpPl/lx6qemWtssx5pAOvLtjKxKSd2cZUuJVu5IMnxjhn59o/Ingsp6OGEhMTzapVq6wpyisC0xuFy8Lpn52xt+0GLhsRZGb9vuM8PjmZTQdPBB1TkXEvv1XNMkBwIwMnjARyU3c0EZHVxpjEjLc7YUSg7Pb7xuwsmDQgBvIVdOVGzXJOm6oKQZOqJZmeIabi/3o3pL8/pkJFP0eNCIKlI4IwS3oFFr7om94IkFjoNNwXM608Y/vhXxk2OZlVu3+hfd1yOcZUOGEvPxRurTuvshoROOLwUWWRrK5D8Hvuf+DjEOOcI3JU+GVzPYpATMVzfRuxZvcvdBuzmPeX7uRimvt2GFXwdETgFTnNZesagTfkYk1j/7HT/H3KehZtTaV59VK2xVSo8NE1Aq/L6eiWcB2J45Jj6j0rF0c5VSlVmPfvuoIv1+7n2Rkb6fXaEh7qVIdBDo6p8OqUT15pI/AKK07EioIjaKJeLj8HIkK/y6vSvm4cz87YyCvzt/KVRTEVulG3jjYCr7DqYvMuPKbeU0L8HARiKvo0rcyIL9fT782l/LV9LYZ0rkfhArERLjpnTjxpzU20EXhJpE/EcmL8Q6h0iitTXRpWoFWtMoyctZkJi3cwd8MhRvZvQpva4Yup0I269bQRqPCJgmPqAWdNcYW7IYXhtZUolJ+R/X0xFU9OSebWd1ZwS8tqDOsRWkxFOGi+UN5oI1Dh5aT4h1A5ZYorEg0pjK+tde2yzB7cgTELtvJO0g5fTEXfxnTNY0yFbtSt58ylf6Xs9Pt5FbH2TnFFInI7zK+tcIFYnux5GV8+0JbSRQow8D+reeCTNaSePJv3WkOgEdah0fMIlMqME9YIIjVFFaHXdv5iGuMX/chrX2+nSMFYnuqlMRVOk9V5BNoI3M4JGywVOS78+24/fJInJq9n9e5f6FAvjpf6NaZq6axjKpR1tBFEIyctaiqVTlqa4T/Ld/PynM0APN6tPne2jicmRkcHdtKsoWjk0cs2KueLiREGtIln3pAOXBFfhmdmbOTG8cvYfvik3aWpTGgjcDOnLGoqlYWqpYvw/l1XMPpPTfkx9Vd6jl3CuK+3cf5imt2lqXR0asjtXDiHrDwiw2cz9eRZnp2xgZnJB2lQsTj/uCGBhKqRjalQf6RrBFbRDbNS2a5fzdtwiKempZB68iz3tK/FIw6JqfACTR+1gi7eKuWTzYlrXRtVpFWtsoyavYnxi3cwJwIxFSp3dI0gnHTxVimfHNavShbOz8j+CXxyTysAbn1nBU9OWc+JM+ftqNbzbG8EIlJNRL4RkU0iskFEBttdU8h08VYpn0DuVKfh2Y6M29Qux5zBHRjYoRaff7+HLqMXMX/jTxYXq2xfIxCRSkAlY8waESkOrAauM8ZszOpndI1A/U7f76iRvO8Yj09KZvOhk/ROqMQzfRpRrlhBu8uKKq5ZLBaRacDrxpj5WT3G0Y1AWUfXZKLOuQu+mIpxC30xFf/XuyH9LteYinBxxQllIhIPXA6syOS+gSKySkRWpaamWl2aciJdk4k6BfLF8NA1dZk1uB21yhVl6Bfr+PO/v2ffL6fsLi2qOaYRiEgxYDLwiDHmRMb7jTETjDGJxpjEuLg46wtUzqNrMlGrTvni/PfeNjxzbUO+33WUrq8u5oPvdpGW5qwZjGjhiKkhEckPzATmGmNG5/R4nRpSv9M1AnfIw99p3y+n+PvUFBZvTSWxRmlGXZ9AnfLFIlRodHPsGoH4Jv8+AI4aYx4J5me0ESjlIqveh1mPQloa5CsY0lqOMYYpa/bz3MyNnD53kYevqcOgjrXJH+uYSQ1XcPIaQVvgDqCTiKz1/9fT7qKUUmGwd6W/CVwA0uDC2ZDWckSE61tUZcHQjnRpVIF/zdvKteOWkLzvWPhr9iDbG4ExZokxRowxCcaYZv7/Ztldl1IqDHYlgUkXMBcTk6e1nLjiBXnj1uaMv6MFR387x3VvLGXkrE2cPncxDMV6l+2NQOXS3pWQ9Irvayj3u020vZ5gRcvrjm8PsQVBYiAmH/R8JSxrOd0aVWT+0I7cdEU1xi/eQY+xi1n2489hKNibbF8jCIVn1whyOm4+2o6rj7bXE6xoe90RXtD/bvsRhk1Zz56jp7ilZXWe7NmAEoXyh/15ooGT1whUsHI6bj7ajquPttcTrGh73dVaQvtHI9bM2tQpx9xHOnBP+5oaUxEibQRuktNx89F2XH20vZ5gefV150HhArEM79WQqfe3pXSRAtzz4Soe/GQNR349a3dprqBTQ26T0zDbacfV57Uep70eq3j1dYfBuQtpvL3oR8Yt3EbRgvk0piIdx55HEApPNwI30bluZaNtP53kicnJrNlzjI714nixX2Oqli5id1m20jUCZb1omusONLWFL/q+uv1oHg+oW8EXU/H0tQ1ZufMo3V5dzIfLNKYiM9oIIi1aDgMMRTTNdWfV1Lz893WB2BjhrrY1mTekA81rlOb/pm3gT+OXsf3wr3aX5ig6NRRJ0TY1EorcTKc4eeols78l2PP3dfL75GDGGCav2c/z/piKwZ3rMrBDLU/FVOg1i+2QzXVbPaNay+Bes1OaZlYb2cAVt9Lfl/SK9X9fp2cdARoAAAplSURBVLxPLiQi3NCiKh3qlePZ6Rv559wtzEw+yD+uT6BJ1ZJ2l2cr77RCO0TT1EikOWE9Iad1gIzHw4fj75vbqSUnvE8uV754Id64rTlv396CI7+epe8bSxg5exNnzns3pkJHBJGU2V6kylxgoxrY07WjaeZ2BJfXv28oe/dOeJ+iRPfGFWldqywvzdrE+EU7mJtyiJH9E2hdu6zdpVlOG0GkBTs14nXZbVStmhMPZSObl79vKFOHunMRViWL5OflGxLo06wyT05Zzy3vLOfWVtUZ1iP4mIqbxi8D4PNBrSNZakRpI1DOkdlG1co5cas3sqHu3evORWiy2aFo64+pGD1/C+8u2cnCTYd54brGdG5YwaZiraVHDSlnS3rFN2dvLvrm4jsN983TRws9AsgaudihWLv3GE9MSmbLTye5tmllnr62IeWKFbzkcYGRwIqdRwFoVbMM4OyRgZ5Qptwp2hfcIxzIpvxyscjerFopZjzUjqFd6jEn5SBdRi9i6g/7cONOc7B0RKCcT/eaVXqhfB5CnGLc6o+p+GHPMa6qH8eL/ZpQpVThPzzGTWsEjj6PQES6A2OBWGCiMWaUzSUpJ9E5cRUQ6ppRiOs/9SoUZ9K9bfhw2S7+MWcLXUcv4okeDbi9VQ1iYqInxM72EYGIxAJbgS7APuB74BZjzMasfkZHBEp5lI1rRnuPnuLvU9eTtO0IV8SXZtT1CdSOK2bJc4eLk9cIWgLbjTE7jDHngM+AvjbXpJRyIhvXjKqVKcKHd7fkXzc2ZetPv9JjbBJvfLOd8xfTcv5hh3NCI6gC7E33/T7/bUppqJv6o8AUT6fhtsRrBGIq5g/tQOfLyvPPuVvo8/pSUvYft7SOcHPCGkFmE22XzFeJyEBgIED16tUjXZNyAs3VUZlxwJpR+eKFePO2FsxJOcRT01Lo+8ZS/tq+JkM616NQ/lhbawuFE0YE+4Bq6b6vChzI+CBjzARjTKIxJjEuLs6y4pSNNFdHOVz3xhVZMKQjNzSvyvhFO+gxNonlO362u6xcc0Ij+B6oKyI1RaQAcDMw3eaalBNE+zkEKioEYio+/msrLqYZbp6wnOFT13PyzHm7Swua7UcNAYhIT2AMvsNH3zPGvJjd4/WoIQ/RcwiUi5w6d4HR87by3tKdlC9eiBf7Neaay5wTU6HXLFZKKYukj6no44+pKJtJTIXVnHz4qFJKRZVATMWQzvWYnXKQzqMX8eUP+x0bU6GNQCmlIqBAvhgGd67LVw+3J75cUR75fC13v/89B46dtru0S2gjUEqpCArEVPxf74Ys33GULqMX8Z9lu0hLc87oQBuBUkpFWGyMcHe7mswb0oHmNUrz1LQN3DxhOT+m/mp3aYA2AqWUskz6mIotP52kx9gk3vzW/pgKbQRKKWWhjDEV/5izhb42x1RoI1BKKRsEYirevr0Fqb+epe8bS3l5zmbOnL9oeS3aCJRSykbpYyre+vZHeoxNYoXFMRXaCJRSymYZYypumrCcEV9aF1OhjUAppRyibZ1yzHmkPX9tV5NPVuyh66uLWbj5p4g/rzYC5Xx6TQLlIUUK5GNE74ZMub8tJQrl5+73V/Hwpz/w869nI/ac2giUswWuSbDwRd9XbQbKIzKLqZi2NjIxFdoIlLPpNQmUh2WMqRj6xTp2Hvkt7M/jhCuUKZW1wDUJAlcp02sSKA8KxFSs2fMLteKKhf33ayNQzha4Rq1ek0B5XGyMcEV8mYj8bm0EyvkccI1apaKZrhEopZTHaSNQSimPs7URiMg/RWSziCSLyFQRKWVnPUop5UV2jwjmA42NMQnAVuBJm+tRSinPsbURGGPmGWMu+L9dDlS1sx6llPIiu0cE6d0NzLa7CKWU8pqIHz4qIguAipncNdwYM83/mOHABeDjbH7PQGAgQPXq1SNQqVJKeZNEIrciVwWIDADuBa4xxpwK8mdSgd0hPmU54EiIP2s3rd16bq0btHa7OLn2GsaYuIw32toIRKQ7MBroaIxJteg5VxljEq14rnDT2q3n1rpBa7eLG2u3e43gdaA4MF9E1orI2zbXo5RSnmNrxIQxpo6dz6+UUsr+EYEdJthdQB5o7dZza92gtdvFdbXbvlislFLKXl4cESillEpHG4FSSnmcJxuB28LuRKS7iGwRke0iMszueoIlItVE5BsR2SQiG0RksN015ZaIxIrIDyIy0+5ackNESonIJP/nfJOItLa7pmCJyBD/5yVFRD4VkUJ215QVEXlPRA6LSEq628qIyHwR2eb/WtrOGoPhyUaAi8LuRCQWeAPoATQEbhGRhvZWFbQLwKPGmMuAK4EHXFR7wGBgk91FhGAsMMcY0wBoikteg4hUAR4GEo0xjYFY4GZ7q8rW+0D3DLcNA742xtQFvvZ/72iebAQuC7trCWw3xuwwxpwDPgP62lxTUIwxB40xa/z/PolvY1TF3qqCJyJVgV7ARLtryQ0RKQF0AN4FMMacM8Ycs7eqXMkHFBaRfEAR4IDN9WTJGLMYOJrh5r7AB/5/fwBcZ2lRIfBkI8jA6WF3VYC96b7fh4s2pgEiEg9cDqywt5JcGQM8DqTZXUgu1QJSgX/7p7UmikhRu4sKhjFmP/AvYA9wEDhujJlnb1W5VsEYcxB8O0NAeZvryVHUNgIRWeCfY8z4X990j8kx7M4BJJPbXHXMr4gUAyYDjxhjTthdTzBEpDdw2Biz2u5aQpAPaA68ZYy5HPgNF0xPAPjn0/sCNYHKQFERud3eqqJf1F683hjTObv7/WF3vfGF3Tl5w7oPqJbu+6o4eKickYjkx9cEPjbGTLG7nlxoC/QRkZ5AIaCEiHxkjHHDRmkfsM8YExh9TcIljQDoDOwMZI+JyBSgDfCRrVXlzk8iUskYc1BEKgGH7S4oJ1E7IsiOP+zuCaBPsImnNvoeqCsiNUWkAL6Fs+k21xQUERF889SbjDGj7a4nN4wxTxpjqhpj4vG95wtd0gQwxhwC9opIff9N1wAbbSwpN/YAV4pIEf/n5xpcstCdznRggP/fA4BpNtYSlKgdEeTgdaAgvrA7gOXGmHvtLSlzxpgLIvIgMBffERTvGWM22FxWsNoCdwDrRWSt/7a/G2Nm2ViTVzwEfOzfedgB3GVzPUExxqwQkUnAGnzTtj/g4MgGEfkUuAooJyL7gKeBUcAXIvIXfI3tRvsqDI5GTCillMd5cmpIKaXU/2gjUEopj9NGoJRSHqeNQCmlPE4bgVJKeZw2AqWU8jhtBEop5XHaCJTKA//1Frr4//2CiLxmd01K5ZZXzyxWKlyeBp4TkfL40lX72FyPUrmmZxYrlUcisggoBlxljDkpIrWA4UBJY8wN9lanVM50akipPBCRJkAl4Kz/4jv4LyL0F3srUyp42giUCpE/YvhjfPn5v4lIN5tLUiok2giUCoGIFAGm4Lsm8ybgeeAZW4tSKkS6RqBUmIlIWeBFoAsw0Rgz0uaSlMqWNgKllPI4nRpSSimP00aglFIep41AKaU8ThuBUkp5nDYCpZTyOG0ESinlcdoIlFLK47QRKKWUx/0/S9HC3taWT4kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "samples_count = 100\n",
    "X, Y = create_samples(samples_count)\n",
    "lr.fit(X, Y.ravel())\n",
    "plt.xlabel('$x_1$')\n",
    "plt.ylabel('$x_2$')\n",
    "theta = np.append(lr.intercept_,lr.coef_ )\n",
    "\n",
    "draw_samples(X, Y, samples_count)\n",
    "draw_border(theta.flatten())\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
